Automated Quality
Some of the participant’s submissions output estimates of sample quality for each processed iris image. The ANSI/NIST-ITL 1-2011 standard requires these estimates to be in the range 0 to 100 and to quantitatively express the predicted matching performance of the sample. Error-reject rate curves show how FNIR can be reduced by discarding the poorest quality samples in the test data. In our case, the quality of a search was set to the minimum quality assigned to the searched image and its enrolled mate.
The figure below demonstrates that FNIR (i.e. the ‘miss rate’) can be reduced by almost 20% by discarding just 1% of the poorest quality searches. Presumably, this 1% involved samples where the subject was blinking, moving, looking off-axis at the moment of capture, etc. The IREX 4 failure analysis found that matching failures for the most accurate matchers over a different dataset were almost entirely due to poor presentation of the iris.
| Dataset: | Operational Dataset 4th pull |
| Samples used: | One eye |
| Enrolled Population: | 1M irides (500K people) |
| Enrollment Method: | One enrollment session per person |
The stacked barplot below shows how sample quality impacts the probability that a search will miss (i.e. fail to return the correct mate). Samples assigned low quality values should be more likely to miss. For Neurotechnology’s matcher, when the assigned value is 0 the probability of a miss is greater than 50%. FPIR is set to \(0.01\).
| Dataset: | Operational Dataset 4th pull |
| Samples used: | One eye |
| Enrolled Population: | 1M irides (500K people) |
| Enrollment Method: | One enrollment session per person |
The sample quality of left and right iris images acquired during the same session are expected to be highly correlated. In addition to having similar capture environments, dual-eye cameras acquire both images at nearly the same instant so poor presentation of the irides at the moment of capture (e.g. blinking or moving at the moment of capture) detrimentally affects both images. For this reason, matching both acquired images vs. matching just one yields only a moderate improvement in accurary. The figure below shows the distribution of qualities with each axis represneting the quality of one of the iris images (left or right) acquired during the same capture session.
| Dataset: | Operational Dataset 4th pull |
| Samples used: | One eye |
| Enrolled Population: | 1M irides (500K people) |
| Enrollment Method: | One enrollment session per person |
The acquisition protocol for OPS4 images has probably improved over time. Better iris cameras and capture environments are likely to have improved the quality of the acquired images. Iris recognition accuracy is highly dependent on the prevalence of very poor quality samples. Misses tend to occur when the subject was blinking, moving, looking off-axis (etc.) at the instant of capture. The figure below shows the prevalence of these very low quality samples in OPS4 for each capture year. Comparatively few images in OPS4 were collected prior to 2014 so results for these images are omitted. An iris sample was deemed to have very low quality if its quality value is among the lowest 2% (i.e. below the 2% quantile) of all images in OPS4.
| Dataset: | Operational Dataset 4th pull |
| Samples used: | One eye |
| Enrolled Population: | 1M irides (500K people) |
| Enrollment Method: | One enrollment session per person |
|
HEADQUARTERS
100 Bureau Drive Gaithersburg, MD 20899 301-975-2000 | |
| How are we doing? Feedback | |